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ABSTRACT 


This thesis presents a discussion of the problems 
[mavorved in estimation of Circular Error Probable (CEP). 
Several estimators are compared ttrough simulation, under 
two models; the power model and the Rayleigh model. Several 
measures of effectiveness are calculated for each of the 
competing estimators. It is found that maximum likelihood 
estimation based on the power distribution verforms well 
for “heavy tailed" distributions; the Rayleigh unbiased 


estimator performs well for most other situations. 
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IL. INGRODUGTION 


Evaluation of a weapons system is frequently an expen- 
sive and time consuming task. Only relatively few tests 
may be allowed because of the high cost of testing. It is 
possible that a good weapon system could be rejected because 
of inefficient utilization of the small amount of data 
available. Hence, efficient use of the limited data is 
reguired to reduce this risk. 

Several authors have considered the measurement of 
weapon delivery accuracy uSing Circular Error Probable 
(CEP). This measure of accuracy 1s defined to be the radius 
of a circle centered at the target which on the average con- 
@amms fifty percent of impact points, that is, the median 
radial miss distance. A functional relationship between 
CEP and the distribution parameters is given by the 
following integral: 


CEP 
F(CEP) = f PGi = 085 1, 
0 


where f(r) is the probability density of radial miss 
distance. 7 

Various parametric models for the distribution of impact 
points about the target have been used in connection with 
such estimations. A discussion of these models 1S given by 


Jordan [lJ]. But, deficiencies in the models have been 





Memed. Specifically, they tend to provide poor fit to the 
Mapper tail” of the radial distributions. That is, they 
provide less than satisfactory explanation of impacts ob- 
served far from the target. A commonly used model is the 
"Rayleigh model", in which impacts in the target plane are 
assumed to be distributed as the unbiased circular normal 
Gistribution. One might anticipate that Rayleigh-based 
CEP estimators might not perform adequately with a large 
proportion of actual test data. Several potential competitors 
to Rayleigh CEP estimators were therefore developed for 
comparison. 

Barr and Jordan [2] developed and proposed the power 
Gistribution model which is based on an infinite mixture of 
Rayleigh radial miss distance distributions. This distri- 
bution has properties compatible with the interpretation of 
Oo in the underlying Rayleigh distribution model; they sug- 
gested the use of maximum likelihood estimates of the param- 
eters of the power distribution which in turn provide a 
maximum likelihood estimate (MLE) of CEP. 

The jackknife 1S a technique which was proposed by 
Quenouille [10] for reducing bias in estimation problems. 

We tried the jackknife procedure to estimate CEP under the 
Rayleigh model. 

The sample median was also used to estimate CEP. Since 

CEP is defined as the median radial miss distance, a natural 


estimator for CEP is the sample median. 





To measure the performance of the estimators, a variety 
of data sets from known distributions, such as uniform, 
Rayleigh, power and Weibull were generated by simulation. 
Samples from the known distributions were generated to 
test the capability of the estimators to handle widely 


disparate situations. 





El.) MAXON LiReGliOoObsESTIMATION BASED 
One THE POV ER-=DISTRIBUTION 
Barr and Jordan [1], [2] have given a discussion of 
CEP accuracy measurement under a power distribution model. 
They showed that estimating the parameters of the power 
distribution by the method of moments may not be adequate; 
moreover it is possible only when the ratio of the squared 


2 does not exceed 1/4 - TT. 


sample mean and variance E fe 
Consequently, they suggested use of the maximum likelihood 
(M.L.) method. 

Analytically closed expressions for the M.L. estimators 
cannot be obtained, the solution of normal equations yielding 
the parameter estimates requires numerical techniques. In 
this section we examine a numerical method to obtain the 
M.L. estimates of the parameters of the power distribution, 


which in turn provide a M.L. estimate of CEP. These estimation 


procedures require use of a computer. 


A. MATHEMATICAL FORMULATION 
The probability density function of the radial miss 
distance R having a power distribution with parameters k and 


ae is 


—- 
aN 





mee. DoF. for radial miss distance is 


: ee ye C a 
FR(rik,3) 1 se ’ 1g a OF K,S > 0 as) 


The CEP is given by 


i 1 


SHEE Pee) = ies - Wicd 


oR 
Because radial miss distance 1s always non-negative, 

one can equivalently consider squared radial miss distance. 

In the case of the power model, this leads to some notational 

Simplification. Let Y denote the squared miss distance Re. 


Then from (1), 


CQ Ca Ae FD Us iene 
(2) 
ms Y,~«-l 
fy(y) = FQ + >) 2 YY ewan > 0 


Here t and x are parameters to be estimated from observed 
impact data. The relationship between CEP and these 
parameters is 


4 
y= ee Sy! 


Q 

a) 

rg 

i] 

ty 
role 


The method of maximum likelihood is based upon the 
likelihood function ¢@(y) which is defined as the joint 


density function of a sample of size nj; 1.e., 


eae 





7 
oe = ea ~~ + 4 SF 


n 
d(y) = wf (y.) = 
= 


Since it 1S easier to deal with sums rather than products 


when maximizing, ¢(y) is transformed into 


L(y) log $(y) 


n 
Meroe ot enero mee) Wit) log(tty,) 


i=l 


There is no loss of generality here because the maximum of 
a poSitive function occurs at the same point as the maximum 


of the logarithm of the function. The normal equations are 





then 
n 
L 
m(k,t) = oh ty) = — + nleg¢— 2 log(t+y,) = Q 
i=l 
and 
g(k,c) = S202 = BE ~ («#1) : ——~— = 0 
C C say © ae 
The solution of m(k,@) = 0 and g(K,C) = Q gives the 


parameter M.L. estimators kK, o As mentioned before, in 


the present case, this requires numerical techniques. 


B. NUMERICAL SOLUTIONS: NEWTON'S METHOD 
This method is a Newton's iteration technique and the 
concept and procedure are discussed in [3]. Newton's method 


provides a statistical criterion for stopping the iterations. 


17 





The pure form of Newton's method is given by the 


following iteration scheme: 


Fos = X. - Pex. Vix 
ja 7% 7 (FOG) (X5) 
where 
Se ieee ], £(X.) = (Sey) Shits 
J oy 2 J JK 92 











Let 


2 2 2 
= amas) . g ity) _ cea? 
OK 16 dK dr 


13 





If D were identically equal to zero, this method would not 


be applicable. Now take 


371 (y) 371 (y) 
2 a ery 
_a*n(y) 3“ (y) 
. ory aie“ 


In order to develop the termination criterion, consider 
the statistic for (for reference see [4]) 


. saline - .-i 
Sr Vie) VER . 
B V (Xs) (Xi) (X.) 


B is asymptotically a Chi-square r. v. with 2 degrees of 
freedom, therefore it might be reasonable to continue the 
iteration until the statistic B becomes less than the 100a 
percentile point of the chi-Square distribution with two 


degrees of freedom, alae 
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TIT. MAXIMUM LIKELIHOOD ESTIMATION BASED 
On THE Mare hLIGH DISTRIBUTICN MODEL 

Suppose impacts, measured ina suitable plane, have 
range (X) and deflection (Y) components of impact that are 
jointly uncorrected bivariate normal distributed with mean 
at the target and common unknown variance ae ineoeteh dilGec- 
tions. That is, for the present development, it will be 
assumed that the joint density of miss distances about the 
target is given by 


22 2 
f(x,y) = (2007) A coal had Seo <Saes ~- 


It is easy to derive the density of radial miss distance. 


Let R = (x+y) 72 and transform to polar coordinates to 


@otain 
2 
R 
Z 1 io aan x 7 
F,(r) = Pier | = f f Re dR dé 
270 0 0 
| i 
_ a, 
Z 


This distribution of the radial miss distance is referred 
to as either the radial normal distribution or the Rayleigh 
feeereribution. 

Then the density function of the radial miss distance, 


f, (r) 1s 
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To find the relationship between CEP and g, we calculate 


CEP = F ~* (5) = j2n 2c = WR >, 


R 
which is a well known relation. Hence, if we want to estimate 
CEP with this model, first we might estimate the population 
standard deviation. The maximum likelihood estimator for 


o is easily found to be 


lea a Se 1s the sample of observed radial miss 


distances. This estimator has a slight bias, however. An 


where rx 


adjusted unbiased estimator for CEP is 


A a a 


: (3) 
. (5(2n+1) ) 


CEP = 1.1774 (/n 
Were the quantity in parenthesis is an unbiasing factor 
for oO. It has been shown by Chapman and Robbins [6] that 
CEP = 1.1848 g for sample size 20 iS a best estimator of CEP 
under the normality and other assumptions stated above. 
However other estimators, while not as efficient as the best 
estimator, may have more robustness under variations in 


the assumed distribution of radial miss distance. 


16 


The term "jackknife" has been used by Miller [9] and 
Quenouille [10] to describe a method of modifying a biased 
estimator to bring it closer to an unbiased condition. 

From (3) we see the M.L. CEP estimator has a bias term of 
fecer —.. We are thus led to try to examine the jackknife 
method s reduce this bias in the estimator. Brillinger([7] 
has provided a discussion of the asymptotic properties of 
estimators obtained with the jackknife method when it is 
applied to M.L.E's. 

The jackknife procedure is based on dividing the data 
into groups, obtaining estimates from combinations of the 
groups, and then averaging these estimates. We apply this 
to our problem, as follows. The maximum likelihood estimator 
for 1s 


n 
ae @* 
0 = i E ioe 


Divide these data into m groups, where ee is the 
sample of n observed radial miss distances and the size of 
each subgroup is k. First estimate the parameter aa based 


on all data 


Nn 
“A ae ; 
fo) Ten ee 


next estimate a based on all data except those in the 2 


group 


16y 


; 

7 

; 
7 

. 

-_ 





ma n 
0 é at 2 ry me ? 1: == doy 24 Cae ae’ Tus 
-1 AY oe 

Bes 


Then compute the pseudo values 


Finally, compute the average of the pseudo values, 
Mm 4 

= y Oo; 

32 _ ae 
m 


and estimate the CEP by 
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IV. THE SAMPLE MEDIAN 


Since CEP is defined as the median radial miss distance, 
a natural estimator for CEP is the sample median (m). By 
definition; the median is the gee percentile of the distri- 


Petron, 1.c., 


Pt(kr< om) > 90.50 and EAR 20m) © 2 ego 0 


Since F(CEP) 0.50 and Ris assumed to have a continuous 


feet? 2 bution, 


Suppose that the n radial miss distance observations of 
the sample are arranged in order of magnitude, Yay! Loy! 
, Fin): We shall consider only the estimates of the 
population median defined as follows: 

(a) If the sample size is odd, andn = 2ptl, the median 
(m) is taken as the (p+1) *2 value. 
(b) If the sample size is even, and n = 2p, the median 


h 


(m) is taken as the midpoint between the pt and 


Ger) ee values or 
iL 
Fey gery) 


ino 





The distribution of the median will tend to be symmetri- 
cal, and should be very nearly normal in almost all popula- 
tions. The distribution of the median in the case of small 
samples from a normal population, whether the sample size 
is odd or even, will tend to be normal [Hoji [1l]). The 
normal form iS approached very rapidly. The value of the 
standard deviations of the median for n even and n odd 


approach the same limit, 
— n e ~ — ~ 
Oa of n = 1.25331 6/ Jn 


where g is the standard deviation in the population sampled. 
We use the order statistics to examine the sample median 

under the Rayleigh distribution model (see Inselmann [12]), 

and find its bias and variance so as to enable one to make 

a comparison with other estimates. If nis even, the dis- 


tribution of sample median is given by the following: 


= _ ” [Oz5n a 
eo .sn)> = "ely (n= (0o5m)) ! ( (Ossm L)E}E(xr) F(x pg shy) 
_ n=O. on | 
tt Sli) eae 
For convenience we let a = 0.5n, and 
C= n!/(n-[(0.5n))!((0.5nJ-1)! = (2a) !/at(a-1)! 


20 





so the sample median is ra: Then we may write the distri- 


bution of sample median under the Rayleigh distribution model, 


2 2 f. 
g(ra) = cl, exp(-5) ] [1-exp(-45) 17-7 [exp (-45) 1° 
O 26 20 20 
2 2 
aa a 
=o 4 fexp(- £4882))) [1 - exp(-,) 177” 
O 20 20 
We will now consider computation of the mean of ra, 
o 2 2 2 
~_ = 
mera) = c f x lexp(-£ $872) j (1-exp(- =~) 1° } ar 
0 6} 20 20 
oo Z a-7-l1 : 
4. —) 
=cf —5 [exp (- - 3 By; Dae 1) *(R2") fexp(--B5) Far 
0 O 20 K=0 20 
a-l o 2 | 
scr (-1)8(93+) + ES exp(- 218) ar 
K=0 0 20 


and make the following transformation 


aL Pee 


0 | 2y/ (at1+K) ’ 


B= fe (acieey7o-i dx 


KK 
II 


K 
I 


ZA 





io obtain, 


aot Kiva: 2 ~ = 
Sa ale) / (ati) = digiiy e ” ady 
K=0 0 


E (ra) 


a-1 (-1)* (70%) 


coms) -/2 - » ein) 
K=0 (a+1]1+K) 


We can adjust the bias term and uSing the same transformation 


as above we may now find the second moment 


eee: 2 a-l K,a-l K 
fe”) =c f Frexp (-L 132) ) 5 os) lal K ae dr 
0 Oo 20 K=0 
K a-l 
ee (- 1) ) 
co K 


K=0 (at1+k) ¢ 


The variances of this sample median can now be computed 


using the well-known formula 


27 





V. SIMULATION METHODOLOGY 


The previous sections have been concerned with a 
development of different estimators for the median of the 
distribution of radial miss distances obtained under a set 
of fixed impact conditions. An analytical comparison is 
difficult to perform for these estimators, so computer 
Simulation was used. This section summarizes the different 
models and estimators discussed in the previous sections, 
and includes an analysis of the results obtained from simu- 
lations. Although the sample problems do not represent actual 
weapon test results, an attempt has been made to generate 
Gata from a wide variety of "realistic" distributions. 
Therefore this Simulation analysis should show relationships 
between our estimators of CEP, and indicate their relative 
robustness, and hence potential for use in a variety of 
applications. The alternative estimators are as follows: 

CEP lz: maximum likelihood unbiased Rayleigh estimator 

CEP 2: maximum likelihood Jackknife Rayleigh estimator, 

with subgroup sample size k = l 

CEP 3: maximum likelihood power estimator under Newton's 

method 

CEP 4: sample median estimator 

For the purpose of our Simulation comparison of these 
four CEP estimators, we generated samples of radial miss 


distances (or, in some cases, Squared radial miss distance) 


23 





from several parent distributions: Rayleigh, uniform, power 
and Weibull distributions. Throughout, we assumed a sample 
size of 20 impact observations. The method of generation 
was by C.D.F. inversion of generated uniform (0,1) values. 
For each parent distribution (hereafter called a "case") 
we replicated estimation of CEP's 500 times. Summaries of 
the sample distributions of 500 values of 5 CEP's were 
developed. 

Eight cases were considered; in each, parameter values 
were used which gave a true CEP of 100. The cases are as 
follows (U is a uniform (0,1) random variable throughout) : 


Case I. (Rayleigh distribution) 
Fp’ (y) =l- Bee 2 with 5 = 107/212 


CEP = V2 ln 2 o 


Generated value: Y = oe ineU = —6§4426.95 tn U 
Case II. (Uniform distribution) 
F.(y) =<-y , with ye (0,200). 
R 200 
CEP = 100 


Generated value Y = 200 U 
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Gase III. (Power distribution) 


= ” ey 
Rg (y) = 2-1 + % 


with ck = 1 and 


al 


C 


+ 


eee = (e2° — oye 


Generated value Y 


Case IV. 


2 == — 
Fe (6'2 ee 


10 


K 


4 


where 20° = 14282.6809 


8 
| 


t+ 


CEP = [100 @ ase In 2}° 


20° 


(Translated Rayleigh distribution) 


2 
e7 (Y-100) /20 


Generated value Y = - in U + LOO 
Cases V-VIII1. (Weibull distributions) 
ae 
V 
4) ay. 
Tey) Lea le 
i 
2wW 


CEP = (v In 2) 


Generated value Y in the table below 


2 





Four sets of parameters (w,v) were selected to give 
CEP = 100 and to provide distributions with varying upper 
tail thickness. One case (with w = 1) gives an exponential 
distribution with mean v, and this corresponds to the Rayleigh 
case. Case I above may be considered also as a Weibull case. 
The parameter pairs used, in order of increasing upper tail 


Size, are given in the following table. 


1 
W 
Case W V Y= (=vind) 
aE 
Vv 2 iowa y = (Siga@ecosaas1 aneu)e 
2 
VI ims oe ino y= (Ga6 os. Oa 
I 1.0 lO vino = aoe osnian 
2 5 
VII G Gay noe oy P= ae ee 


vit ay. 10°*/1n2 Ne (=3. 62360 6.21 cy 
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VI. SIMULATION RESULTS 


As mentioned above, we tested four CEP estimators from 
eight different cases. Major comparison meaSures include 
bias, varlance and mean square error. Others are included 


in the tables that follow. 


a 





GSE I. RAYLEDGH. DISTRIBUTION 


Y = — 14426295 1n U 

MOE CEP dee CEP 2 CEP 3 CEP 4 
MEAN Sr9 3 DIO po 20 107 
Sib BEV Pr. 66 ied 619 | ea Saeed 
MSE Sion 0:3 S63 7 Gromer / 1S 4 9 
GED MEAN FD > Mee. 25 O67 100 
VARIANCE Ps'o-.'0 3 fe Fo 1 a4 5 1500 
SOEMrY VAR O22 Ougl2 Oiggias 0.36 
SKEWNESS Or 0.14 O04 9 Oe 
KURTOSIS 0.10 0.085 0.307 0.34 
RANGE Tiles @ Ones 83 ..60 ZZ 
MINIMUM oe 8) 6 Trae? S3ew 4 Cie T 

-10 QUANTILE 84.82 84.80 Sno 9 58.4 

B25 Gia he 92509 Gr .6/ O24-5 

220 Sp ar oa. 2 9eGs o)3) SNS 105 

an 107.49 107.34 104.00 liei2 

70 ira oS @ ar Oro Wane. 32 roo 
MAXIMUM 1G. 6 3 30.16 Leyes 269 


* CEP 1: maximum likelihood unbiased Rayleigh estimator. 


CEP 2: maximum likelihood Jackknite Rayleigh estimator, 
with subgroup sample size k = l. 


CEP 3: maximum likelihood power estimator under 
Newton's Second method. 


CEP 4: sample median estimator. 
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CASE II. UNIFORM DISTRIBUTION 


y = (200 vu)” 


MOE CEP 1 CEP 2 CEP 3 CEP 4 
MEAN WGmo. OG 2 ol 72 OZ 
pilD DEV 2.67 eels Les 41.7 
MSE EOS: 3 2 HO 7 ies Oks 306763 1744.0 
GED MEAN 26. 0D 21s OM a Uae. 
VARIANCE 97. 6S 969 2 PS Wo) 1740 
COEFF VAR On 20 Cra liad 0.40 
SKEWNESS = 10meze = Orzo eto CeO) 
RURTOSIS Omet 0.60 5.54 me Yc) 
RANGE Gia S G65 5 26.0 jes) 
MINIMUM Ss a oe aaa | oF. U et |) 

mo QUANTILE 84.19 Ore 2o ae 4). 7 

me SD G0". TD NOT Sia. 30 742 

0 96.84 96.74 94.26 102 

<a ge. Or, 916 NQIZt 7) 2S LO 132 

se 0 TOS66 108.40 107.62 158 
MAXI MUM ion 6 0 PZiowro 2 NZ 00 194 


A 


* some CEP = 0.0 
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MOE 


MEAN 
STD DEV 
MSE 
GED MEAN 
VARIANCE 
COEF VAR 
SKEWNESS 
KURTOSIS 
RANGE 
MINIMUM 
~10 QUANTILE 
aS 
. 50 
Sigs) 
m0 


MAXIMUM 


CASE.111. 


on 


ie 


mre 


POWER DISTRIBUTION 


CEP 


2a 
FS) I 
83000 
De as 3) 


66515. 


1 


fic 


v2 


30 


2 


33 


90 


62 


7 


Fo 


SZ 


76 


we 
08 
ie, 
noo 
20 
03 
ap 


seo 


Lie 


124 


206. 


ia ial 


84 


107 
140 
Liz 


354 


~40 


. 76 








CASE 


MOE 
MEAN 
oD DEV 
MSE 
GED MEAN 
VARIANCE 
GoEF VAR 
SKEWNESS 
KRURTOSIS 
RANGE 
MINIMUM 
mao QUANTILE 
sao 
2 OO 
. 
a0 


MAXIMUM 


iV. TRANSLATSeD RAYEREGH DISTRIBUTION 


Y= = 1426226809 In U + 100 
CEP 1 CEP 2 CEP 
Oe? 9 ae 9 m5 % 
Ll. 56 11.0518 122 
PSee7 4 134.09 176 
90.1.2 On9)..1..1 94 
Lear 0 13405 Ss 
Orel 2 Oigple2 0 
OES OvenlkS - © 
OjgelO 0.08 4 
TO. 7 70 25 135: 
67.44 67.46 7 
84.81 84.78 8] 
92.04 920.0 & 7 
990468 9) ..498 96 
iO iiaeg2.8 LO@aas2 1:03 
114.56 1 Vaan 9 AIL 
LSiGeel9 L3G: L:As3is, 


on 


3 
85 


64 


m2'6 
6 
74 
3 
a5 
209 
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CASE V. WEIBULL DISTRIBUTION ww @ 
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Y = (- 144260504.1 In U)” 
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GSE VI. WEDBUGE DESTRIBUBION w=1.5 
2 
Y = (- 1442695.04 ln u) > 
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COMPARISON OF ESTIMATORS 


USING MEAN SQUARED ERROR AS A CRITERION: 


MSE[CEP] = uae + [E(CEP) - CEP]? 

CASE NO. DISTRIBUTION CEP 1 CEP 2 CEP 3 CEP 4 
I RAYLEIGH 136 136 165 1549 
can UNIFORM 109 107 368 1744 
er 1 POWER 2460 83,000 444 2279 
IV TRANSLATED RAY. 134 134 ey 1524 
V Vee Oli w=. 2. ZanG 228 379 Boz 
VI WEIBULL w = 1.5 158 164 32 664 
VII WEIBULL w = .5 6246 6230 1153 11956 
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VII. CONCLUSION 


In general, we found the maximum likelihood power 
estimator CEP 3is better than moment method power estimator, 
in that it does not have the limitations and exists for all 
parent distributions we considered. Using minimum mean 
Squared error as a criterion, the estimators based on the 
power model have inferior performance in the "shallow tailed" 
cases, but were best in the heavy tailed cases, this is 
especially true for the w = .1 Weibull case, in which it 
has less bias than its competitors and has relatively small 
variance. 

We found the Rayleigh based CEP estimators (CEP 1) EO 
be possibly the best overall, especially for the shallow 
tailed cases. The maximum likelihood jackknife Rayleigh 
estimator (CEP 2) had no better performance than the unbiased 
Rayleigh estimator. 

The value of the sample median estimator (CEP 4) can 
be found for small samples with such rapidity that in certain 
cases the time saved may compensate for the accuracy lost. 

In practice, when analysts find several impact points 
comparatively far away from target, they should choose 


maximum likelihood power estimator (CEP 3). Otherwise they 


could choose Rayleigh based CEP estimator (CEP 1). 
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